Add versioned benchmark JSON schema validation (v1) - #10

🟡 Changes recommended

The schema/tests do not currently enforce the required protocol field per Issue #3 acceptance criteria, so the validation guarantees are incomplete.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Thanks for taking this on. The schema direction is useful, but this needs four focused fixes before merge:

1. Please remove the newly added top-level protocol field from both the fixture and schema. The issue guidance explicitly defines the protocol through the existing required fields (kind, token counts, trials, budget, buffer, and seed) and asks not to change the producer format solely to add a key.
2. Add jsonschema to the test extra in pyproject.toml. In a clean project environment, pytest -q currently fails during collection with ModuleNotFoundError: No module named 'jsonschema'.
3. Add the schema compatibility policy to results/README.md: additive optional fields remain compatible within v1; removing/renaming fields or changing their semantic meaning requires a new schema version.
4. Run ruff format on tests/test_benchmark_schema.py; ruff format --check . currently fails on the final assertion.

Verified locally: ruff check . and mypy src pass; formatting and pytest fail for the reasons above. Please rerun the complete CONTRIBUTING.md check list after updating.

Thanks for addressing all four prior findings. I reran the complete CONTRIBUTING.md suite locally; 34 tests and the package build pass.

There is one remaining schema-compatibility issue before approval:

• RandKVConfig and both CLIs allow buffer_size=0, but the schema uses minimum: 1, so it rejects a valid benchmark result. Please change this to minimum: 0.
• budget is an integer in RandKVConfig and argparse, but the schema declares it as number, which accepts fractional values the producer cannot emit. Please use type: integer with minimum: 1.

Please add a small regression test showing a zero buffer validates and a fractional budget does not. Everything else from the previous review is resolved.

Approved. The latest commit resolves the remaining schema compatibility findings: zero-sized recency buffers validate, budgets are constrained to positive integers, and both cases have regression coverage. Verified locally with 36 tests plus Ruff, formatting, mypy, package build, and diff checks; the hosted Python 3.10/3.13 matrix and GitGuardian also pass.